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Abstract 

We show that the stochastic Schrodinger equation for the filtered 
state of a system, with linear free dynamics, undergoing continual non- 
demolition measurement or either position or momentum, or both to- 
gether, can be solved explicitly within a class of Gaussian states which 
we call extended coherent states. The asymptotic limit yields a class of 
relaxed states which we describe explicitly. Bellman's principle is then 
applied directly to optimal feedback control of such dynamical systems 
and the Hamilton Jacobi Bellman equation for the minimum cost is de- 
rived. The situation of quadratic performance criteria is treated as the 
important special case and solved exactly for the class of relaxed states. 

PACS numbers: 07.55. Ge, 42.50.Lc, 03.65. Ta, 05.45.Mt 



1 Introduction 

Quantum noise was originally developed to model irreversible quantum dynam- 
ical systems, where it played an external and secondary role, however, the real- 
ization that it could be measured and the results used to influence the system 
evolution has had a profound effect on its physical status EQ>E1>E]- The great 
leap forward since then has been made by experimentalists who have made the 
practical implementation of quantum state estimation and adaptive feedback 
control a reality. With this, has come new problems that have received intense 
interest in the physics community [4-10]. 

In this paper, we wish to treat the problem of how to describe the quan- 
tum evolution of a system with linear free dynamics when we perform non- 
demolition measurements of, typically both, canonical position and momentum. 
The problem where position measurements only are made has been of historical 
importance. In this situation, the model is the one considered by Ghirardi, Ri- 
mini and Weber JT], who also obtained the asymptotic form for the state. The 
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asymptotic solution, with explicit reference to the stochastic Schrodinger equa- 
tion within the Ito formulation, was first given by Diosi ^5|> see a l so Belavkin 
and Staszewski UI] ■ Essentially, the solution to the stochastic Schrodinger equa- 
tion could be understood as an randomly parameterized Gaussian state. The 
parameters being mean position, mean momentum and a complex inverse vari- 
ance. We shall show that the same class of states, which we term extended 
coherent states, suffice for the stochastic Schrodinger equation describing simul- 
taneous monitoring of position and momentum. 

The problem of optimal quantum feedback control can then be tackled at 
this point. Bellman equations have been derived previously for the optimal 
cost of controlling a qubit system UJ. In fact, the general problem can be 
understood as a classical control problem on the space of quantum states Ql] if 
one exploits the separation of quantum estimation component from the control 
component: here we may construct a, typically infinite dimensional, Hamilton 
Jacobi Bellman theory and are then faced with the problem of finding a sufficient 
parameterization of states for particular situation. In the case of non-demolition 
position and momentum measurements, we have that the extended coherent 
states offer a sufficient parameterization. The quadratic performance problem 
is the important special case and has been treated by Doherty and Jacobs [Ta] 
for feedback from measuring one quadrature of a Bosonic mode. We show that 
this problem is solvable when both canonical observables are measured. 

1.1 Stochastic Schrodinger Equation 

Consider a quantum system evolving with free Hamiltonian H while undergo- 
ing continual diffusive interaction with several independent apparatuses, each 
coupling to the system in a Markovian manner with coupling operator Lj for 
the j-th apparatus. (The {Lj} do not generally need to be either commuting or 
self-adjoint.) The state, ip t , of the system continually updated using the output 
of the apparatuses, will then satisfy a stochastic Schrodinger equation of the 
type [SI, UM, 

i# t ) = ivo dt - \ ( L l L i - 2X > (*) L i + A ? w) dt 

3 

+ J2(L j -X j (t))\i> t )dW} j) . (1) 

3 

where Xj {t) = Re {ij) t \Lj jp t ) and {W^ } is a multi-dimensional Wiener process 
(?) (k) 

with dWY'dWl ' = 8 jk dt. This equation was first postulated in the context 

of filtering by Belavkin where the apparatuses are separate Bose fields and the 

(1) 

W t are innovations processes obtained by de-trending the output processes. 

The stochastic Schrodinger equation for measurement of canonically conju- 
gate observables, q and p, has been derived from first principles by Scott and 
Milburn ^7]. They considered a discrete time model with simultaneous mea- 
surement of position and momentum by separate apparatuses, and considered 
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the continuous time limit of progressively more imprecise and frequent mea- 
surements. Taking L\ = ■J^Q an d £2 = y^P an< ^ denoting the innovations 

by Wjp — Wt and — Wt, their particular stochastic Schrodinger equation 
reads as 

l#t) = {^H-1{q-{q) t f- j(p-{p)t) 2 )\i> t )dt 

~(q- (q) t ) \r/, t ) dW t + yf (P - (p) t ) |Vt> dW t . (2) 

The equation involves the expectations (q) t = (ip t \qip t ) and (p) t = (ip t \pip t ) 
and is therefore non-linear in the state ip t . Here the constants k m _2 s _1 and k 
N~ 2 s~ 3 are positive and describe the measurement strength for the two appara- 
tuses. In general, K and k has units of inverse variance of position, respectively 
momentum, per unit time. In |19|. the limiting procedure was revisited and, as 
an alternative to increasingly imprecise measurements, one could use increas- 
ingly weak interaction between the apparatuses and the system. The scaling 
between the imprecision of measurement, or weakness of interaction with the 
apparatus, and the rate at which the discrete measurements is made must be 
such as to allow a general central limit effect to take place. In principle, it is 
possible, to set up the apparatuses to obtain desired values of k and k. 

The purpose of |17j was to consider nonlinear dynamics, however, we shall 
only deal with quadratic Hamiltonians of the type H = H (/, v) 

h = -^f + \hn 2 -f q + vp. (3) 

Here / and v are external fields which will later be replaced with control func- 
tions. We shall show that it is possible to find a general solution for the stochas- 
tic state ip t , with initial condition being that we start in a coherent state, realized 
as a random wave function taking values in a special class of wave functions, 
termed extended coherent states. 



2 Extended Coherent States 

Let L 2 (R) be the Hilbert space of square integrable functions of position coor- 
dinate x with standard Schrodinger representation of the canonical observables 
q and p. By an extended coherent state, we mean a wave function tp (q, p, rf) , 
parameterized by real numbers q, p and a complex number r; = if + irj" where 
rf > 0, taking the form 

( rf \ ^ 4 f V - 2 Pi 

(x\ip(q,p,T]))=l—J cxpj--(x-g) + i-xj . (4) 

When rj is real (rf' = 0), the vectors are just the well-known coherent states [T8| 
The distribution of the canonical variables in extended coherent state ip (q, p, rf) 
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is Gaussian with characteristic function 

(cxp {irq + isp}) qpv = exp {irq + isp - i (C qq r 2 + 2C qp rs + C pp s 2 ) j , (5) 
where 

r 1 r W r - h% ( < t*\ 
qq ~ 7' qp ~ ~ ~2r/' w ~ T I rf )' 

The mean values of the position and the momentum in an extended coherent 
state are evidently (q) = q and (p) = p respectively. We have that C qq is the 
variance of q, C pp is the variance of p, while C qp — \ {qp + pq) — (p) (q) is the 
covariance of q and p. 

2.1 Derivation of the Characteristic Function 

To establish (J5J , let us first recall that coherent states may be constructed from 

creation/annihilation operators a^ 1 = —y/rfq± -=p by identifying %j) (q,p, rf) 

2 /7/ v //' 

1 ^ 1 



as the eigenstate of a with eigenvalue a = — yrfq 7=P- I n particular, if 

2 ih\J rf 

fl denotes the zero-eigenstate of a~ then 

where D a = exp{aa + — a* a"} is a Weyl displacement unitary. Next observe 
that we may obtain extended coherent states from coherent states by the simple 
application of a unitary transformation: 

^ (q, P, rf + W) = V ipiq, p, t]') 

with V = exp | — jrj" (q — ?) 2 |. (This transformation is, in fact, linear canon- 
ical.) We may introduce new canonical variables q' and p' by q' = V'qV = q 
and p' = V^pV — p — \%rj' (q — q). We note that exp {irq + isp} = D z where 

z — ——fryfrf s + i—p=r and 
2 vV 

V^D„V = exp {irq' + isp'} = D w e i iHr >"^ 



displacement operators ^B] and their fl-state averages, we find 



where w = —-hy/rfs+i—= (r — ^Hrf's). Using well-known properties for Weyl 



(exp{irq + isp}) q ^ v = {£l\D\y^ D z VD a ti) 

= (n\DlD w D a n)e^" e ' 



h\ w \ 2 e h ihr i" i s 



and substituting in for a and w gives the required result. 
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2.2 Weyl Independence 

We say that the canonical variables are Weyl independent for a given state ( • ) , 
not necessarily pure, if we have the following factorization 

(exp {irq + isp}} = (exp {irq}} (exp { isp}} 

for all real r and s. If the state possesses moments to all orders, then Weyl 
independence means that symmetrically (Weyl) ordered moments factor ac- 
cording to (: f(q)g(p) :) = (/(?)) (g(jp)), for all polynomials f,g. By inspec- 
tion, we see that coherent states leave the canonical variables Gaussian and 
Weyl-independent. However, the rp ^ extended states do not have this Weyl- 
independence property. 



3 Stochastic Wave Function 

We now return to the equation for the conditioned state i/j t . Let (X) t — 
(ip t \ X \ip t }, for a general operator X, then we have the following stochastic 
Ehrcnfest equation 



d(X) = jl {[X,H]) - - A ([[X,q],q]) J ([[X,p] ,p})\ dt 

+ ^({Xq + qX) - (q) (X)) dW t + ^{{Xp + pX} - (p) {X)) dW t . (7) 

For X = q,p, we find 

d(q) = [ — (p) +v) dt + V2^C (q, q) dW t + VlRC (q,p) dW t , 
\m J 

d(p) = (-Hfj,(q) + /) dt + V2^C(q,p) dW t + V2kC(p,p) dW t . (8) 

where C (q, q) = (q 2 ) - (q) 2 , C (jp,p) = (p 2 ) - (pf , and C (q,p) = \ (pq + qp) - 
(p) (q). In the following, we wish to investigate the dynamical evolution of the 
random state ip starting from an initial coherent state. It turns out however 
that we do not remain within the class of coherent states: if we did, then q and p 
would remain Weyl-independent and, in particular, C (q,p) would vanish, along 
with the noise term in the (p)-equation of JSJl above and this would lead to an 
inconsistent system of equations. Fortunately, it turns out that it is possible to 
think of ip as evolving as a random state taking values amongst the extended 
coherent states. Explicitly, we make the ansatz that the state tpt takes the form 

ip t = i ) (9t,Pt,Vt) ( 9 ) 

where q~t and p t are real- valued diffusion processes satisfying and r) t is a complex- 
valued deterministic function. Our assumption that we start from a coherent 
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state is equivalent to asking that 77 (0) = a~ 2 > 0, with a having the interpre- 
tation as the initial dispersion in position. 

We shall now show that q, p satisfy the diffusion equations JSJ, while 77 
satisfies the Riccati equation 

-7-77 = 2k + i2u — i (kh 2 + i—] ri 2 . (10) 
at 2 \ m J 

3.1 Consistency with the Statistical Evolution 

Let r, s be fixed real parameters and set D = exp {irq + isp}. We shall investi- 
gate the evolution through the characteristic function 

G t = (1> t \D\1> t ) = (D) t . 
Observing that [D,q] = hsD, [D,p] = —hrD we find 

dG={±<fiD + DP) - i jf(qD + Dq) + (ifs + ivr - + G \ dt 

+ ^1 ((Dq + qD) - (q) G) dW + ^| ((Dp + pD) - (p) G) dW . 

The identity e iri}+ls P = e ¥ rsh e ^e' sp = e -¥™K e i s p e irq (Baker Campbell Haus- 
dorff formula) then allows us to compute that 



and likewise 



(Dq) = [q + i (C m r + C qp s) - -sh ) G, 



(pD) = [p + i (C qp r + C pp s) + -rh ) G, 



(Dp) = (p + i (C qp r + C pp s) - -rh ) G. 



Hence 



dG — — {p + i (C qp r + Cp p s)} G dt — ishfj, {q + i (C qq r + C qp s)} G dt 

( . , . nh 2 s 2 kh 2 r 2 \ „ 

+ [tfs + ivr G dt 

V 4 4 J 

+iV2~H (C qq r + C q ps) GdW + iV2k (C qp r + C pp s) G dW . (11) 



G 



Under our ansatz we should also have, by the Ito rule, 



dG dG ,_ 1 d 2 G , ,_, a d 2 G , ,_ ,_, 1 d 2 G , ,_ n2 

+—dn' + —d" 

drf drj" 

= irGdq + isGdp - ^r 2 G (dqf - rsG (dqdp) - ^s 2 G (dp) 2 



hi' 



1 , 

n 

rs 

2rf 4 j]' 



-rs [1 

2t/ 8 



V 



1/2 



dtf 



1 h 2 r," 2 



d v ". 



(12) 



Equating the coefficients of i|ll|) and (|12|l gives the system of equations 



1 



dq=[—p + v)dt + V2nC qq dW-" + V2kC qp dW 



dp = {-hfj,q + f) dt + V~2KC qp dW + V2kC pp dW 
(dqf - —dr]' 



if 

h 2 



1 ^ , ^ 2 , 

— C„„dt H — — at, 



m ' 2 



1 ft 2 ?? 



(dp) 2 + _ 1 - ' + - — ^^77" = -27^ dt+ — dt, 



2 ry 



rs : (dgdp) + —7 dr?' - — drj" = —C pp dt - h[i C qq dt. 
2i]' z 2rf m 

The first two of these agree exactly with {8J, while the next three are entirely 
consistent with the pair of real equations 



d_ 

Tit 



7/ = 2k +—7/77" -\kh 2 (rj' 2 
m 

d ,. _ h 



n 



t/2\ 



(13) 



v " = 2 Ll - — {r ] ' 2 -r 1 " 2 )-kh 2 r ] 'r 1 ". 
ox zin 



Together, they are equivalent to the single complex Riccati equation (JT 



3.2 Asymptotic States 

The Riccati equation (|10|) is to be solved in the half plane 77' > of physical 



solutions and has the unique, globally attractive, fixed point 

2 



K + Ifl 



\ mh 

(Here If- denotes the complex root having positive real part.) 



(14) 
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In the case of a harmonic oscillator of frequency w, we have a = — - — > 

h 

and we may achieve a coherent state (77^ real) as the limit state if we tune 

9 9 ~ 2muj 
the measurement strengths such that k = m ui k. in this case, rj^ = — - — , 



, h 

corresponding to a coherent state with position uncertainty — \ . 

V 2mw 

Otherwise the limit state will be an extended coherent state. 

We should remark that «/— corresponds to the squeezing parameter s in- 

V K 

troduced in |17j to describe the bias in favor of the q or p coupling. 

4 Optimal Quantum Feedback Control 

We fix a terminal time T > and let {f t : < t < T} and {v t : < t < T} be 
prescribed functions which we refer to as control policies. Let ip t = tp (qt,Pt, Vt) 
be the solution to the stochastic Schrodinger equation with time-dependent 
free Hamiltonian H = H(f t ,v t ) and initial state being an extended state 
ip (q~o,po, r/ ) at time to somewhere in the time interval [0, T]. 

We wish to grade the control policies {ft} and {vt} over the time interval 
[to,T] and do so by assigning a cost J = J [{ft} , {v t } ; to, T; qo,Po, Vol taking 
the general form 

J [{ft}, {vt} ; t ,T; g , Po,Vo\ = / H s ', fs,v s ;q s ,p s ,r] s ) ds + g (q T ,PT,VT) ■ 

J t 

(15) 

Here £ is a function of time, the current control policy values, and current state 
parameters. The function g, known as a target or bequest function in control 
theory, is a function of the state parameters at termination. We assume that 
both are continuous in their arguments. 

The cost J will vary from one experimental trial to another, and must be 
thought of as a random variable depending on the measurement output. The 
aim of this section is to evaluate the minimum average cost over all possible 
control policies, which we denote as 

S (t ,T;q ,po,T] ) = min E { J [{/ t } , {v t } ; to, T; q ,Po, Vo]} ■ 
{/*}>{"*} 



4.1 Bellman Optimality Principle 

For simplicity, let us write z = (g, p, rj) and u = (/, v) and S = S (to! Zt ), etc. 
Taking to < to + At < T, we have that 

{rta+At ~| 
/ t (s; u s ;z s ) ds + J [{u t } ; t + At, T; z + Az] I 
Jt J 
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where Az = z t — z to is, of course the random change in the state parameters 
from time to to to + At. We have that 



*o+At 



i (s; u s ; z s ) ds — £ (t , u to , z tQ ) At + o (At) 



up to terms that are small of order in At. Likewise, assuming that S will be 
sufficiently differentiable, 



S {t + At- zo + Az) 



6 wz ) + 



S (to;z ) 

dS_ 
+ Ifrf 
1 cfS_ 

+ 2 dq 2 

d 2 S 



dS_ 
~dt 



drf 
~dt 



At 



dS 

, oz 
dS 

At+ IT 
i dq 

dS 



1 a i d2s 



Az + o (At) 







i \ A as 

-p + v t )At+ — 
m J Op 



(-h/iq + f t ) At 



drf_ 
~dt 



At 



[2kC, 



</</ 



«-^a i a 1 9 2 S 



[2nC qp + 2kCp P ] At 



dqdp 



2x/^ [C 2 qq + C 2 pp ] C qp At + o (At) 



(On the right hand side, we are evaluating at to, qoiPo, Vo-) 

The Bellman principle of optimality (201, see also |23 f° r instance, states that 
if {itj} is an optimal control policy exercised over the time interval [io,T] f° r a 
given start state at time to, then if we operated this policy up to time to + At 
then the remaining component of the policy will be optimal for the control 
problem over [t + At, T] with start state being the current (random) state at 
time to + AT. If we assume the existence of such an optimal policy, then, within 
the above approximations, as At — > + , we are lead to the partial differential 
equation (Hamilton Jacobi Bellman equation, or just Bellman equation) for 
S = S(t;q,p, V ) 



n dS / _ dS dS 
= -5- + U t; q,p, 77; — , — 
Ot \ Oq op 



S r 2 - 1 O ^ ^ 

W [ qq + qpl + dm 



KKj \Cq 



d^drf_ 
drf~dt 



OS drf 

W [ qp 



kCL 



(16) 



where we introduce 



H(t;q,p,myq,yp) :=m ™ (j^ p + v ^ +yp(^ h M + /) +^ 9, p,»7)j ■ 

It should perhaps be stressed that the derivation of this equation is entirely 
classical. The key feature of the Bellman equation is that the minimum is now 



9 



taken pointwisc: that is we look for the optimal scalar values /, v at a single 
instant of time. The equation is to be solved subject to the terminal condition 
lim f ^ T - S (t,q,p,r)) = g(q,p,j]). 

In principle, once a minimizing solution f* = f (t; q,p, rf) , v* — v* (t; q,p, rf) 
is known, it may be used as a Markov control for closed loop feedback: that is, 
the control policies are taken as these functions of the current state parameters. 

The Bellman equations arising in quantum feedback control have so far 
proved to be highly nonlinear and prohibitively hard to solve as a rule. Our 
equation (|16fl is no exception, however, the nonlinearities are in due to the r\ 
variable. We remark that if we assume that we start off in a state relaxed at the 
equilibrium value r\ = 77^, then the coefficients of the 77', 77" derivatives vanish 
exactly, and we may take the covariances C qq , C qp and C pp at their relaxed value 
determined from JHJ) evaluated at the asymptotic value 77^. As the relaxation 
time is typically small, we may justify this for large times T in comparison. 
This ignores any 77-transient contribution to the cost, but at least opens up the 
possibility of solving the Bellman equation and finding optimal Markov con- 
trol policies. We give the fundamental class of interest, quadratic performance 
criteria, next. 



4.2 Linear Quantum Stochastic Regulator 

We consider the following quadratic control problem not involving any costs 
on the 7/ parameter. In particular, we make the assumption that the starting 
state is an asymptotic state (77 = 77^) and so we ignore 77 as a variable. We set 
x = {q,p) and u — (f,v) and take the specific choices 

£(t,u,x) = -x'A t x + -u'E t u, 
9 (x) = -x'Rx, 

where At, E t and R are 2x2 symmetric matrices with E t being invertible. The 
free Heisenberg equations are linear and can be written as it = F t Xt + M t u. The 
control problem is now essentially the same as the classical stochastic regulator 
|21| . In this case we introduce a dual variable y to x and obtain the TL- function 

H(t,x,y) = mm{£(t,u,x) + y' (F t x + M t u)} 

U 

= ^x'A t x + y'F t x + min |^u'.E t tt + y'M t u^ 
with the minimum attained at 

u* = -E- x M[y 

and we find 

H (t, x, y) = -x'A t x + y'F t x - -y'M t E- x M' t y 
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Seeking an ^-independent solution, the Bellman equation (|16f) reduces to 
dS 1 d 2 S 



= — +H(t,x,VS) + -K, 



dt ' ' 2 dxidxj 

Here if is the matrix of the second order coefficients in (|16[) and these will be 
determined by the covariances © determined at the asymptotic value 77^. As 

is well known |21|. the solution takes the form S (t, x) — —x'T, t x + a t where Et 
satisfies the matrix Riccati equation 

= -E t F t - F t 'E t + XtMtE^M'Zt ~ A t , S T = R, 



dt 

while at satisfies 



^ = -tr a T = Q. 

at 



The optimal control policy is therefore given by 

u* (t, x) = -E^M' t VS = Et X M' t T, t x. 



4.3 Commentary 

The sufficiency property of the extended coherent states means that the results 
above are of importance to the corresponding filtering problem. Indeed this 
allows us to implement a quantum analogue of the Kalman filter for state es- 
timation amongst the class of extended coherent states. The Kalman filter is 
of considerable conceptual and practical importance in classical control theory 
and plays a crucial role in optimal feedback control. In fact, the matrix Riccati 
equation occurring in linear stochastic regulator also appears in a dual formula- 
tion as a Kalman filtering problem |21| . Unfortunately, the solution to the fully 
parameterized Bellman equation, that is, when we do not start from the equi- 
librium value 77 = "q^, seems to be disappointingly difficult even in the linear 
regulator example as the matrix K will be quartic in r\. (Such difficulties seem 
to be sadly the norm in applications to optimal quantum control as a whole, so 
far.) The control problem is however tractable for the class of relaxed coherent 
states and corresponds to the linear regulator model for quadratic performance 
and this at least gives us some insight into possible applications. 

Acknowledgment: The author is indebted to Luc Bouten for a critical 
reading of the article. 
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